Imaging device and imaging method

ABSTRACT

An imaging device includes: an image sensor that images an object to be imaged through polarizing plates arranged to have a different polarization direction for each pixel in a pixel group that includes a plurality of pixels corresponding to each of points of the object to be imaged; a pixel selecting unit that selects a pixel having a lowest brightness for each of the pixel group corresponding to each of the points; and an image output unit that outputs a captured image of the object to be imaged that is generated from pixels selected by the pixel selecting unit.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation application of InternationalApplication PCT/JP2016/056095, filed on Feb. 29, 2016 and designatingthe U.S., the entire contents of which are incorporated herein byreference.

FIELD

The present invention relates to an imaging device and an imagingmethod.

BACKGROUND

Biometric authentication has become widespread recently. The biometricauthentication is used to verify whether an operator of a computer orthe like is a proper operator, for example, to authenticate an operatorof a cash dispenser at a store. In this case, an imaging device images,for example, palm veins of an individual. When the vein pattern based onthe image obtained by the imaging device matches with a vein patternthat has been registered in advance, the individual is verified. Thecash dispenser accepts operations of only operators verified by thebiometric authentication.

Imaging devices for various purposes including biometric authenticationsystems emit infrared lights to irradiate a palm held toward an imagingdirection of the imaging device to capture an image. A part of theinfrared lights is reflected on a surface of the palm, but a partthereof passes through the skin and is absorbed by hemoglobin of veinsinside the hand. Therefore, by imaging with an infrared-light sensitiveimage sensor, a vein pattern inside a palm can be acquired in additionto surface information of the palm.

For example, such a imaging device has a self-illuminating device thatemits an infrared light to irradiate a palm through a ring-shaped lightguide, and acquire a palm image by an imaging system constituted of alens and an image sensor. The acquired palm image includes a backgroundimage such as of a ceiling and a wall, and this can cause adisadvantageous influence on cutting a palm out of a whole image orsignal extraction of a vein pattern in an image processing stagedepending on conditions.

Therefore, in an actual situation, a method in which images are takenunder two conditions of turning on and off the self-illuminating device,and a difference image thereof is used as a palm image is applied. Therespective images taken when the self-illuminating device is on and offinclude the same background, and the background is canceled in thedifference image, thereby acquiring a palm image only.

Ambient light, such as sunlight and room light, not only illuminates abackground, but also illuminates a palm. Also the illumination by suchambient light or external light on a palm can cause disadvantageousinfluence on signal extraction of a vein pattern, but is canceled by thedifference image, and an image that is illuminated only by theself-illuminating device can be obtained (For example, refer to PatentDocument 1).

Patent Document 1: Japanese Laid-open Patent Publication No. 2014-078857

However, in the above technique, external-light-bearing capacity islimited, and when external light is strong like outside, there areproblems as explained below.

Because it is a precondition that the brightness at a palm portion ofthe respective images of when the self-illuminating device is on and offare not saturated, to obtain an accurate difference image, if theexternal light becomes intense, the exposure when taking images with theself-illuminating device turned on or off is to be decreased. Thisresults in a decreased difference between the respective images takenwith the self-illuminating device turned on or off, and it thus becomesdifficult to extract a vein pattern accurately. Therefore, an intensitylimit is set for external light as a use environment, there is adisadvantage that some measures are necessary to be taken such as usingan appropriate cover to make an artificial shade when used outside.

When the external light intensity is high, using only external light forillumination can be considered as another option, but a locally highbrightness region (halation, etc.) can occur depending on an incidentangle of the external light to a palm, and it can cause adisadvantageous influence on signal extraction of a vein pattern.Therefore, there is a problem that accurate extraction of a vein patternbecomes difficult.

SUMMARY

According to an aspect of the embodiments, an imaging device includes:an image sensor that images an object to be imaged through polarizingplates arranged to have a different polarization direction for eachpixel in a pixel group that includes a plurality of pixels correspondingto each of points of the object to be imaged; a pixel selecting unitthat selects a pixel having a lowest brightness for each of the pixelgroup corresponding to each of the points; and an image output unit thatoutputs a captured image of the object to be imaged that is generatedfrom pixels selected by the pixel selecting unit.

The object and advantages of the invention will be realized and attainedby means of the elements and combinations particularly pointed out inthe claims.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory and arenot restrictive of the invention.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating one example of an imaging deviceaccording to a first embodiment;

FIG. 2A is a perspective view illustrating one example of the imagingdevice according to the first embodiment;

FIG. 2B is a cross section taken along line I-I illustrating one exampleof the imaging device according to the first embodiment;

FIG. 3A is a diagram illustrating one example of a structure of a colorimage sensor;

FIG. 3B is a diagram illustrating one example of a structure of amulti-polarization image sensor in the imaging device according to thefirst embodiment;

FIG. 4 is a diagram illustrating one example of a pixel structuringmethod of the color image sensor;

FIG. 5A is a diagram illustrating one example of a pixel structuringmethod of the multi-polarization image sensor of the imaging deviceaccording to the first embodiment;

FIG. 5B is a diagram illustrating one example of the multi-polarizationimage sensor of the imaging device according to the first embodiment;

FIG. 6A is a diagram illustrating one example of an act of reflectedlight from an object to be imaged in the imaging device according to thefirst embodiment;

FIG. 6B is a diagram illustrating one example of an act of reflectedlight from an object to be imaged in the imaging device according to thefirst embodiment;

FIG. 7 is a perspective view illustrating regular reflection and changesin a polarization direction on an object surface;

FIG. 8 is a schematic diagram illustrating a relationship betweenregular reflection and a polarization direction from a direction of thenormal of an object surface;

FIG. 9 is a diagram illustrating a palm when illuminating with intenseexternal light;

FIG. 10 is a diagram illustrating one example of a brightnessdistribution of an image including regular reflection out of reflectedlight from an imaging object in the imaging device according to thefirst embodiment;

FIG. 11A is a diagram illustrating one example of a brightnessdistribution of an image in the imaging device according to the firstembodiment when a self-illuminating device is on under intense externallight;

FIG. 11B is a diagram illustrating one example of a brightnessdistribution of an image in the imaging device according to the firstembodiment when the self-illuminating device is off under intenseexternal light;

FIG. 11C is a diagram illustrating one example of a brightnessdistribution of a difference image in the imaging device according tothe first embodiment when the self-illuminating device is on and offunder intense external light;

FIG. 12 is a flowchart illustrating one example of imaging processing inthe imaging device according to the first embodiment; and

FIG. 13 is a diagram illustrating another example of a method ofstructuring the color image sensor.

DESCRIPTION OF EMBODIMENTS

Embodiments according to the present application are explained below,referring to the accompanying drawings. Following embodiments explain animaging device that is applied to a vein authentication apparatus toauthenticate a person based on characteristics of veins of the person asan example, but are not intended to limit the disclosed technique. Thedisclosed technique is applicable to most imaging devices that acquire,for example, biometric information of a subject from a difference image.The respective embodiments can be combined within a scope of not causinga contradiction. Furthermore, like reference symbols are assigned tolike parts throughout the embodiments, and explanation of components andprocessing that have already been explained is omitted.

First Embodiment

(Imaging Device According to First Embodiment)

In the following, an imaging device according to a first embodiment isexplained, referring to FIG. 1 to FIG. 5B. FIG. 1 is a block diagramillustrating one example of an imaging device according to the firstembodiment. FIG. 2A is a perspective view illustrating one example ofthe imaging device according to the first embodiment. FIG. 2B is a crosssection taken along line I-I illustrating one example of the imagingdevice according to the first embodiment.

As illustrated in FIG. 1, an imaging device 100 according to the firstembodiment includes a control unit 110 that is a processing device suchas a microcomputer, an image sensor 120, and a light emitting device130. As illustrated in FIG. 2A, in the imaging device 100, an imagingunit 1 is housed in a casing 100 a, and is covered by a cover 100 bhaving infrared transparency. The imaging device 100 images a palm thatis held over the cover 100 b.

As illustrated in FIG. 2B in a cross section taken along I-I of FIG. 2A,the imaging unit 1 of the imaging device 100 has the image sensor 120,the light emitting device 130, a lens unit 3, and a light guide 4 on asubstrate 2, and has the control unit 110 on a control substrate 5.Light emitted by the light emitting device 130 is projected upwardthrough the light guide 4 as illumination light from the cover 100 b.This light illuminates a palm held over the cover 100 b, and isreflected on or absorbed in the palm. The image sensor 120 takes animage of the palm that is held over the cover 100 b through lenses ofthe lens unit 3.

The image sensor 120 is a multi-polarization image sensor in which onepixel is structured, for example, by each of four image sensors having apolarizing plate that differs from others in a polarization direction.In the following, a pixel structuring method in the image sensor 120 isexplained. Multi-polarization image sensors are structured based on thesame idea as color image sensors, and are structured by replacing colorfilters in color image sensors with polarizing plates.

FIG. 3A is a diagram illustrating one example of a structure of a colorimage sensor. For example, as illustrated in FIG. 3A, color imagesensors corresponding to respective three primary colors of R (Red), G(Green), B (Blue) are arranged in the Bayer array. The arrangementpattern illustrated in FIG. 3A is repeated in a sensor area of the colorimage sensor.

FIG. 3B is a diagram illustrating one example of a structure of themulti-polarization image sensor in the imaging device according to thefirst embodiment. In the first embodiment, the multi-polarization imagesensor is structured based on the method of structuring the color imagesensor illustrated in FIG. 3A. The pixel of B illustrated in FIG. 3A isreplaced with a pixel in which a polarizing plate with a latticedirection of 0° is arranged. Similarly, the pixel of G on the upperright side illustrated in FIG. 3A is replaced with a pixel in which apolarizing plate with a lattice direction of 45° is arranged. Similarly,the pixel of R in FIG. 3A is replaced with a pixel in which a polarizingplate of a lattice direction of 90° is arranged. Similarly, the pixel ofG on the lower left side illustrated in FIG. 3A is replaced with a pixelin which a polarizing plate of a lattice direction of 135° is arranged.

In this example, a wire-gird polarizing plate is assumed to be used, andas for a transmittance direction and a reflection direction of therespective polarizing plates, as illustrated in FIG. 3B, it passes whenthe polarization direction is perpendicular to the lattice direction,and it reflects when the polarization direction matches with the latticedirection. As the polarization directions of transmittance andreflection are determined according to the direction of lattice, in thisspecification, a term, polarization direction of a polarizing plate isalso used instead of a term, direction of lattice. Moreover, inprinciple, absorbing polarizing plates can also be applied, and in thatcase, reflection in the above description is replaced with absorption.

Thus, as illustrated in FIG. 3B, polarizing plates having differentpolarization directions are arranged for respective four pixels, andthis arrangement is repeated in the entire sensor area similarly to thecolor image sensor, thereby structuring the multi-polarization imagesensor.

FIG. 4 is a diagram illustrating one example of a pixel structuringmethod of the color image sensor. In the color image sensor, one pixelis expressed with four pixels having different characteristics. Asillustrated in FIG. 4, in a broken-line-surrounded section 11, one pieceeach of R and B, and two pieces of G are included, and this isconsidered to structure one pixel. An output of G is a mean value ofoutputs of two pieces of G. Next, a broken-line-surrounded section 12shifted rightward by one column is considered as a next pixel to theright. The broken-line-surrounded section 12 shares one piece of G and Bwith the broken-line-surrounded section 11, but uses different piecesfor the other G and R. As described, pixels in a horizontal direction inFIG. 4 are structured. Similarly, pixels in a vertical direction in FIG.4 are structured.

The multi-polarization image sensor also structures pixels similarly tothe color image sensor. FIGS. 5A and 5B are diagrams illustrating oneexample of a pixel structuring method of the multi-polarization imagesensor of the imaging device according to the first embodiment. Asillustrated in a broken-line box in FIG. 5A, one pixel is structuredwith four polarization directions of 0°, 45°, 90°, and 135°. A nextpixel to the right is structured as in a broken-line box shiftedrightward by one column as illustrated in FIG. 5B. In both a horizontaldirection and a vertical direction, by shifting the pixel structure byone, an image having polarization characteristics of four directions canbe acquired while maintaining the number of output pixels, similarly tothe color image sensor.

To explain functions and effects of the first embodiment, an act ofreflected light from an object to be imaged is first explained. FIG. 6Aand FIG. 6B are diagrams illustrating one example of an act of reflectedlight from an object to be imaged in the imaging device according to thefirst embodiment. In a palm image when illuminated with external light,a dynamic range of brightness increases by a local high-brightnessregion, and there are disadvantageous influences, such as decrease in anentire signal level, and disappearance of a minute signal covered by alocal high-brightness region, to palm authentication. Occurrence of alocal high-brightness region is caused mainly by regular reflection ofillumination light from a palm surface appearing in an image. Moreover,while external light has random polarization, components of light thatis regular-reflected on an object surface are biased such thatS-polarization component is intense.

As illustrated in FIG. 6A, incident light to an object is not justreflected on a surface of the object, but is also reflected such amanner that it enters the inside of the object, repeats diffusion andreflection, and then comes out of the surface. Light that has entered tothe inside is diffused inside a skin tissue of a person, and repeatsdiffusion and reflection to be emitted from a surface, and it has littledirectivity and isotropically diffuses as illustrated in FIG. 6A.

Furthermore, as illustrated in FIG. 6B, even with minute unevenness on asurface of an object, diffuse reflection in which light is reflected invarious directions occurs. The diffuse reflection repeats reflection,and the polarization direction is thus disordered, and even if incidentlight is linearly polarized, diffuse-reflected light becomes random. Inaddition to the diffuse reflection, a part of light is regular-reflected(mirror reflection) on a surface of an object. Unlike the diffusereflection, the regular reflection has a strong directivity to adirection of mirror reflection.

Moreover, the regular reflection also has distinctive polarizationcharacteristics. The reflectivity of light on an object surface differsdepending on a polarization direction, and P-polarized light in which anelectric field vibrates in a plane of incidence is likely to passthrough a boundary, and has low reflectivity. On the other hand, thevibration of an electric field is perpendicular to a plane of incidence,that is to say that S polarization light parallel to a boundary has highreflectivity. Therefore, even if incident light is randomly polarized,regular reflection light is to be light having strong S polarizationcomponent. This phenomenon, that is, the regular reflection and a changeof a polarization direction on an object surface, is illustrated in aperspective view in FIG. 7.

As illustrated in FIG. 7, the polarization direction of regularreflection has strong S polarization in which an electric field vibratesin a direction perpendicular to a plane of incidence that is determinedby the normal of an object surface and a direction of incident light.FIG. 8 is a schematic diagram illustrating a relationship betweenregular reflection and a polarization direction with a plan view from adirection of the normal of an object surface. As described above, thepolarization direction of light that is regular-reflected on an objectsurface is mostly the S polarization, that is, a direction perpendicularto a plane of incidence. Therefore, when illumination light of randompolarization enters from a direction of (A) in FIG. 8, the polarizationdirection of light that is regular-reflected on the object surface ismostly 90°. Similarly, the regular reflection of illumination lightentering from a direction (B) is mostly 45°, and the regular reflectionof illumination light entering from a direction (C) is mostly 135°.

Even when an incident direction of illumination varies, if a polarizingplate to shield the S polarization light with respect to the directioncan be always applied, regular reflection light can be significantlyreduced even with illumination light of random polarization, such as sunlight.

In the first embodiment, a function of reducing this regular reflectionlight to the sufficiently practical level is achieved by usingproperties that the adjacent four pixels of the multi-polarization imagesensor pass polarization light of respective different directions. Thus,even when illumination by external light that has random polarizationand, the incident direction of which cannot be controlled is used, aneffect of acquiring an excellent palm image without a localhigh-brightness region can be expected.

FIG. 9 illustrates a palm when illuminating with intense external light,such as sun light, and illustrates light that is imaged on a sensorsurface of the image sensor 120 by the lens unit 3 in this example. Thatis, a palm image in FIG. 9 is an input signal to the image sensor 120,and this light is subjected to intensity modulation by the polarizingplates of the multi-polarization image sensor, and is converted into animage signal of the image sensor 120.

For example, when illumination light of external light enters from adirection (B) in FIG. 9 to the palm and a local high-brightness regionis generated at a lower right part of the palm image, out of the fourdifferent polarization directions of the multi-polarization imagesensor, the polarizing plate of the direction of 135° passes regularreflection light the most, and the brightness of the pixel becomes high.Those of the directions of 0° and 90° are the second highest, and thebrightness of the pixel of the direction of 45° is the lowest.Furthermore, when the illumination of external light that forms thelocal high-brightness region at the lower right part enters from adirection of (A) in FIG. 9, out of the four different polarizationdirections of the multi-polarization image sensor, the polarizing plateof the direction of 90° reflects regular reflection light the most notto let it pass. That is, the intensity of light that has passed throughthe polarizing plate of the direction of 90° is the lowest, and isconsidered as substantially diffuse reflection light.

As described, by choosing entire pixels following a rule to use a pixelof the lowest brightness per group of four pixels, an excellent palmimage in which regular reflection light is removed can be acquired.

Explanation is returned to FIG. 1, FIG. 2A, FIG. 2B. The light emittingdevice 130 is a light source, such as an LED (light emitting diode),that emits random polarization light to irradiate a palm, which is anobject to be imaged, held over the imaging device 100. Hereinafter, thelight emitting device 130 can be referred to as self-illumination. Onthe other hand, ambient light such as sunlight and room light can bereferred to as external illumination. Moreover, the self-illuminationlight and external illumination light can be referred to as illuminationlight, collectively.

The control unit 110 includes a lighting control unit 110 a, an imageacquiring unit 110 b, a pixel selecting unit 110 c, and an image outputunit 110 d. The lighting control unit 110 a controls lighting on and offof the light emitting device 130.

The image acquiring unit 110 b instructs the lighting control unit 110 ato turn off the light emitting device 130. The image acquiring unit 110b then acquires an image by causing the image sensor 120 to capture animage (hereinafter, external light image in some cases) including a palmheld over the imaging device 100 and a background in a state in whichthe light emitting device 130 is turned off. The external light is lightother than light by the self-illumination, such as outdoor natural lightor indoor room illumination light.

Furthermore, the image acquiring unit 110 b instructs the lightingcontrol unit 110 a to turn on the light emitting device 130. The imageacquiring unit 110 b then acquires an image by causing the image sensor120 to capture an image (hereinafter, self-illumination image in somecases) including a palm held over the imaging device 100 and abackground in a state in which the light emitting device 130 is turnedon.

Subsequently, the image acquiring unit 110 b acquires a difference imagebetween the external light image and the self-illumination image.

The image acquiring unit 110 b determines whether the brightness of apredetermined portion (for example, a central portion of the image) ofthe external light image is equal to or higher than a first threshold,or whether the brightness of a predetermined portion (for example, acentral portion of the image) of the difference image between theexternal light image and the self-illumination image is lower than asecond threshold. The image acquiring unit 110 b uses the external imageas an image subject of processing of pixel selection when the brightnessof the predetermined portion of the external image is equal to or higherthan the first threshold, or when the brightness of the predeterminedportion of the difference image between the external image and theself-illumination image is lower than the second threshold.

On the other hand, the image acquiring unit 110 b uses the differenceimage as an image subject of processing of pixel selection when thebrightness of the predetermined portion of the external image is lowerthan the first threshold and the brightness of the predetermined portionof the difference image between the external image and theself-illumination image is equal to or higher than the second threshold.The image acquiring unit 110 b uses the external light image as an imagesubject of processing of pixel selection even if the brightness of thepredetermined portion of the external image is lower than the firstthreshold, if the brightness of the predetermined portion of thedifference image between the external light image and theself-illumination image is lower than the second threshold.

The pixel selecting unit 110 c selects a pixel, the brightness of whichis the lowest per pixel group that is constituted of adjacent fourpixels as one group, throughout the whole image subject of processing.The adjacent four pixels are four pieces of pixels that are arranged ina lattice-like structure. The image output unit 110 d generates one palmimage by connecting pixels selected by the pixel selecting unit 110 c.

(Brightness Distribution of Image Including Regular Reflection)

FIG. 10 is a diagram illustrating one example of a brightnessdistribution of an image including regular reflection out of reflectionlight from an imaging object in the imaging device according to thefirst embodiment. A regular reflection region at a lower right portionin a figure expressing a palm appears in a brightness distribution graphillustrated at a lower part of FIG. 10 as a locally high brightness.

As illustrated in FIG. 10, the brightness distribution of the entirepalm image is that a local high brightness caused by regular reflectionlight entering through the lenses is superimposed on the brightnessdistribution of a signal image that is generated by diffuse reflectionlight. Therefore, by avoiding the regular reflection, continuous imagesby the diffuse reflection light can be acquired. By using the smallestvalue out of brightness outputs of the four polarization directions, theregular reflection light by the random polarization illumination can besignificantly removed, and inconvenience due to biased brightnessdistribution in one image can be reduced, and the continuity of pixelsin image data can be expected enough.

(Switch from Self-Illumination to External Illumination According toFirst Embodiment)

FIG. 11A is a diagram illustrating one example of a brightnessdistribution of an image in the imaging device according to the firstembodiment when the self-illuminating device is on under intenseexternal light. FIG. 11B is a diagram illustrating one example of abrightness distribution of an image in the imaging device according tothe first embodiment when the self-illuminating device is off underintense external light. FIG. 11C is a diagram illustrating one exampleof a brightness distribution of a difference image in the imaging deviceaccording to the first embodiment when the self-illuminating device ison and off under intense external light. Switch from theself-illumination to the external illumination according to the firstembodiment is explained, referring to FIG. 11A to FIG. 11C.

When the intensity of the external light is not high like that ofindoors, the brightness of an image when the self-illumination is OFF islow as expressed by a broken line in FIG. 11B, and therefore, thedifference image between the external light image and theself-illumination image is to be an image with sufficient difference inbrightness as expressed by a broken line in FIG. 11C. However, in anenvironment with intense external light, the difference image betweenthe external light image and the self-illumination image is to be animage in which the brightness of an image when the self-illumination isOFF is high as expressed by a solid line in FIG. 11B, and that hasinsufficient difference in brightness as expressed by a solid line inFIG. 11C.

Specifically, when the intensity of the external light becomes high, thebrightness is to be high in both of the case in which theself-illumination is ON illustrated in FIG. 11A and the case in whichthe self-illumination is OFF illustrated in FIG. 11B, and exposure timeis therefore shortened so as not to saturate the brightness.Accordingly, a brightness difference in the difference image between theexternal light image and the self-illumination image becomes small. Asdescribed, when the brightness difference is small, an influence oflight noise or circuit noise increases, and a disadvantageous influenceoccurs in image processing for extraction of a vein pattern of a palm.

Therefore, in the first embodiment, a method is switched, by thresholddetermination, from a method of acquiring a palm vein pattern by adifference image to a method of acquiring a palm vein pattern only bythe external light image. Specifically, in the first embodiment, whenthe brightness of, for example, a central portion of the external lightimage illustrated in FIG. 11B becomes equal to or higher than the firstthreshold, the method is switched from the method of acquiring a palmvein pattern by a difference image to the method of acquiring a palmvein pattern only by the external light image. Moreover, in the firstembodiment, when the brightness of the difference image illustrated inFIG. 11C becomes lower than the second threshold, the method is switchedfrom the method of acquiring a palm vein pattern by a difference imageto the method of acquiring a palm vein pattern only by the externallight image.

That is, in the first embodiment, a palm is imaged by theself-illumination indoors, or the like when the intensity of theexternal light is lower than predetermined intensity. As the intensityof the external light increases due to usage at a window or outside,appropriate exposure time to acquire a difference image between thosewhen the self-illumination is ON and OFF becomes short. Therefore, inthe first embodiment, the method is switched from the method ofacquiring a palm vein pattern by a difference image to the method ofacquiring a palm vein pattern only by the external light image withrespect to a predetermined condition.

(Imaging Processing in Imaging Device According to First Embodiment)

FIG. 12 is a flowchart illustrating one example of imaging processing inthe imaging device according to the first embodiment. As illustrated inFIG. 12, the imaging device 100 first determines whether to image a palm(step S11). When the imaging device 100 images a palm (step S11: YES),the processing proceeds to step S12. On the other hand, when the imagingdevice 100 does not image a palm (step S11: NO), step S11 is repeated.

At step S12, the imaging device 100 turns off the self-illumination.Subsequently, the imaging device 100 acquires an external light image ofthe palm (step S13). Subsequently, the imaging device 100 determineswhether the brightness of the external image (for example, at apredetermined portion) is equal to or higher than the first threshold(step S14). When the brightness of the external light image (at apredetermined portion) is equal to or higher than the first threshold(step S14: YES), the imaging device 100 shifts the processing to stepS21. On the other hand, when the brightness of the external light image(at a predetermined portion) is lower than the first threshold (stepS14: NO), the imaging device 100 shifts the processing to step S15.

At step S15, the imaging device 100 turns ON the self-illumination.Subsequently, the imaging device 100 acquires a self-illumination image(step S16). Subsequently, the imaging device 100 turns OFF theself-illumination (step S17). Subsequently, the imaging device acquiresa difference image between the external light image acquired at step S13and the self-illumination image acquired at step S16 (step S18).

Subsequently, the imaging device 100 determines whether the brightnessof the difference image (for example, at a predetermined portion) islower than the second threshold (step S19). When the brightness of thedifference image (for example, at a predetermined portion) is lower thanthe second threshold (step S19: YES), the imaging device 100 shifts theprocessing to step S21. On the other hand, when the brightness of thedifference image (for example, at a predetermined portion) is equal toor higher than the second threshold (step S19: NO), the imaging device100 shifts the processing to step S20.

At step S20, the imaging device 100 uses the difference image as animage subject of processing. On the other hand, at step S21, the imagingdevice 100 uses the external light image as an image subject ofprocessing. When step S20 or step S21 is completed, the imaging device100 shifts the processing to step S22.

At step S22, the imaging device 100 selects a pixel having the lowestbrightness per pixel group from the image subject of processing. Theimaging device 100 then generates an output image from pixels selectedat step S22 (step S23). Subsequently, the imaging device 100 determineswhether to end the imaging processing (step S24). When determining toend the imaging processing (step S24: YES), the imaging device 100 endsthe imaging processing, and when determining not to end the imagingprocessing (step S24: NO), the processing is shifted to step S11.

At step S19, when the brightness of the difference image (for example,at a predetermined portion) is lower than the second threshold (stepS19: YES), the imaging device 100 can skip performing the processing atstep S21 to step S24, and can shift the processing to step S11.Alternatively, the determination can be performed only based on thefirst threshold at step S14, and the processing at step S19 can beomitted.

Alternatively, upon acquiring either of the “external light image” orthe “self-illumination image”, the processing of image selection andgeneration of an output image at step S22 to step S23 can be performedfor the acquired image, and the other image acquisition can be omitted.

Alternatively, turning OFF of the self-illumination at step S12 andturning ON of the self-illumination at step S15 can be switched, andacquisition of an external light image at step S13 and acquisition of aself-illumination image at step S16 can be switched. That is, a “firstimage” and a “second image” can be the “external light image” and the“self-illumination image”, respectively, or can be the“self-illumination image” and the “external light image”, respectively.

In the first embodiment, while using external light for illumination, anexcellent palm image in which a local high-brightness region, such asglare, is removed can be acquired.

Although the control unit 110 of the imaging device 100 performs thevarious kinds of processing illustrated in the flowchart of FIG. 12 inthe first embodiment, it is not limited thereto, and by connecting theimaging device 100 and an external control device (not illustrated), theexternal control device can control the imaging device 100 to performthe various kinds of processing.

Second Embodiment

In a second embodiment, the pixel structuring method is based on anotherpixel structuring method, unlike the first embodiment based on the pixelstructuring method of the color image sensor illustrated in FIG. 4. FIG.13 is a diagram illustrating another example of a method of structuringthe color image sensor. Although only an output signal of B is availablefor a pixel corresponding to B22 in FIG. 13, a mean value of four pixelsof adjacent R11, R13, R31, and R33 is calculated as R22, and a meanvalue of four pixels of G12, G23, G32, and G21 is calculated as G22, toobtain output signals of R, G, B of the pixel corresponding to B22.Furthermore, although only an output signal of G is available for apixel corresponding to G23 in FIG. 13, a mean value of two pixels ofadjacent R13 and R33 is calculated as R23, and a mean value of twopixels of B22 and B24 is calculated as G23, to obtain output signals ofR, G, B corresponding to G23.

In the second embodiment, pixels of the multi-polarization image sensorare structured in a similar manner as in the pixel structuring method ofthe color image sensor. That is, the method of replacing respectivecolor filters of R, G, B in the Bayer arrangement with respectivepolarizing plates of 0°, 45°, 90°, and 135° is the same as the firstembodiment. Moreover, other points are also the same as the firstembodiment.

Pixels can be structured by using various publicly-known pixelstructuring methods other than that of the first and the secondembodiments according to a demanded image quality.

The respective components illustrated in the first to the secondembodiment can be modified or omitted within a range not departing fromthe technical scope of the imaging device according to the disclosedtechnique. Furthermore, the first to the second embodiments are merelyone example, and not only the modes described in the section ofdisclosure of the invention, but also other modes in which variousmodifications and improvements are made based on knowledge of a personskilled in the art are also included in the disclosed technique.

According to one example of the disclosed technique, for example,accurate extraction of a vein pattern is enabled.

All examples and conditional language provided herein are intended forthe pedagogical purposes of aiding the reader in understanding theinvention and the concepts contributed by the inventors to further theart, and are not to be construed as limitations to such specificallyrecited examples and conditions, nor does the organization of suchexamples in the specification relate to a showing of the superiority andinferiority of the invention. Although one or more embodiments of thepresent invention have been described in detail, it should be understoodthat the various changes, substitutions, and alterations could be madehereto without departing from the spirit and scope of the invention.

What is claimed is:
 1. An imaging device comprising: an image sensor that images an object to be imaged through polarizing plates arranged to have a different polarization direction for each pixel in a pixel group that includes a plurality of pixels corresponding to each of points of the object to be imaged; a pixel selecting unit that selects a pixel having a lowest brightness for each of the pixel group corresponding to each of the points; and an image output unit that outputs a captured image of the object to be imaged that is generated from pixels selected by the pixel selecting unit.
 2. The imaging device according to claim 1, further comprising: a light emitting device that emits illumination light having random polarization to irradiate an object to be imaged; a lighting control unit that controls on and off of the light emitting device; and an image acquiring unit that acquires a first image and a difference image, the first image captured by imaging the object to be imaged by the image sensor in a state in which the light emitting device is controlled to be off, the difference image between the first image and a second image captured by imaging the object to be imaged by the image sensor in a state in which the light emitting device is controlled to be on, wherein the pixel selecting unit selects a pixel having a lowest brightness for each of the pixel group corresponding to each point of either image of the first image and the difference image according to a brightness of any one of the first image and the difference image.
 3. The imaging device according to claim 2, wherein the image acquiring unit cancels acquisition of the second image and the difference image when the brightness of the first image is equal to or higher than a first threshold, and the pixel selecting unit selects a pixel having a lowest brightness for each of the pixel group corresponding to each point of the first image.
 4. The imaging device according to claim 2, wherein the image acquiring unit acquires the second image and the difference image when the brightness of the first image is lower than a first threshold, and the pixel selecting unit selects a pixel having a lowest brightness for each of the pixel group corresponding to each point of the difference image when the brightness of the difference image is equal to or higher than a second threshold, and selects a pixel having a lowest brightness for each of the pixel group corresponding to each point of the first image when the brightness of the difference image is lower than the second threshold.
 5. An imaging method comprising: imaging an object to be imaged through polarizing plates arranged to have a different polarization direction for each pixel in a pixel group that includes a plurality of pixels corresponding to each of points of the object to be imaged; selecting a pixel having a lowest brightness for each of the pixel group corresponding to each of the points; and outputting a captured image of the imaging object that is generated from pixels selected at the selecting. 